Hybrid HTTP and UDP content delivery

ABSTRACT

A hybrid HTTP/UDP delivery protocol provides significant improvements for delivery of video and other content over a network, such as an overlay. The approach is especially useful to address problems (e.g., slow startup times, rebuffering, and low bitrates) for HTTP-based streaming. In general, the protocol has two phases: an HTTP phase, and a UDP phase. In the HTTP phase, the client sends an HTTP GET request to a server. The GET request contains a transport header informing the server that the client would like to use UDP-based transfer over the protocol. The server may refuse this mode and continue in ordinary HTTP mode, or the server may respond by sending an empty response with header information informing the client how to make the connection to enter the UDP phase. In the UDP phase, the client initiates a connection and receives the originally-requested content over UDP.

BACKGROUND Technical Field

This application relates generally to data communication over a network.

Brief Description of the Related Art

Distributed computer systems are well-known in the prior art. One suchdistributed computer system is a “content delivery network” or “CDN”that typically is operated and managed by a service provider. Theservice provider typically provides the content delivery service onbehalf of third parties (customers) who use the service provider'sshared infrastructure. A distributed system of this type is sometimesreferred to as an “overlay network” and typically refers to a collectionof autonomous computers linked by a network or networks, together withthe software, systems, protocols and techniques designed to facilitatevarious services, such as content delivery, application acceleration, orother support of outsourced origin site infrastructure. A CDN serviceprovider typically provides service delivery through digital properties(such as a website), which are provisioned in a customer portal and thendeployed to the network.

Overlay networks of the type described provide HTTP-based streaming froma server in the network, to a client that receives a video stream andplays it back to a screen. Clients may be running on a desktop computer,mobile device (smart phones, tablets), set-top-boxes, smart televisionsor appliances, or any other network-connected devices. For a typicalHTTP-based streaming use case, the client resolves a name (e.g., a CDNhostname) through DNS to an IP address and makes a TCP connection to theserver associated with that address. Once established, the client sendsan HTTP GET request for the desired content, e.g., a media fragment. Theserver responds with the content data in a standard HTTP response body.The connection is kept open for further requests and responses.

Standard approaches using the Hypertext Transfer Protocol (HTTP) make arequest over HTTP protocol (on top of a Transmission ControlProtocol/Internet Protocol (TCP/IP) connection) and receive the responsecontaining the requested data in the HTTP response (over the same TCP/IPconnection). Alternative streaming approaches use a protocol on top ofUser Datagram Protocol (UDP) or TCP for both requests and responses(including control messages). Still other approaches tunnel someprotocol through an HTTP session.

Today many end users experience problems with HTTP-based streaming, suchas slow startup times, rebuffering, and low bitrates. Connectivity ofthese users often shows large enough bandwidth for high-quality video,but the round-trip times and packet loss characteristics between theserver and the client negatively affect streaming performance, primarilybecause standard TCP-based implementations operate inefficiently on suchnetworks.

There is thus a need to provide new techniques to improve videostreaming quality. The disclosure herein addresses these and otherproblems associated with the relevant prior art.

BRIEF SUMMARY

A hybrid HTTP/UDP delivery protocol provides significant improvementsfor delivery of video and other content over a network, such as anoverlay. The approach is especially useful to address problems (e.g.,slow startup times, rebuffering, and low bitrates) for HTTP-basedstreaming.

In general, the protocol has two phases: an HTTP phase, and a UDP phase.In the HTTP phase, the client sends an HTTP GET request to a server. TheGET request contains a transport header informing the server that theclient would like to use UDP-based transfer over the protocol. Theserver may refuse this mode and continue in ordinary HTTP mode, or theserver may respond by sending an empty response with header informationinforming the client how to make the connection to enter the UDP phase.In the UDP phase, the client initiates a connection and receives theoriginally-requested content over UDP.

The hybrid approach differs from conventional HTTP-based streaming byadding a separate channel for content transmission over a UDP protocol.Once a client connects to a server, the client sends an HTTP GET requestfor the desired content, e.g., a media fragment, and includes an HTTPheader indicating that it wishes to use a UDP connection for thetransfer of the body of the response. The client may also include anypreferences about the UDP delivery (e.g., forward error correction(FEC), or other parameters). The server, if provisioned to use theprotocol, responds with an empty body (Content-Length: 0) answer. Theresponse preferably also includes various headers to prevent proxies onthe way from caching the response. In the response headers, the clientfinds information on how to connect to the same server over UDP protocol(or also other extra alternate server references, e.g., for multi-sourcecontent acquisition). This information typically includes an IP address,a UDP port number, optionally some other special port number (toidentify which flow the communication belongs to, because there may bemany such connections on the same server's UDP port), version, and tokenT. Once this information is received from the server, the clientinitiates a UDP connection by sending a SYN UDP packet to the serverbased on the information provided in the HTTP response headers (IPaddress, UDP port number, other port number, etc.). This requestpreferably also includes a token T from the HTTP response. Using thetoken, the server thus can verify that the client sending the UDP packetsaw the previous HTTP response and that such SYN packet should beprocessed. The SYN preferably also contains the client's InitialSequence Number (ISN). The server responds with a SYN_ACK packet, whichincludes the acknowledged ISN value from the client's SYN packet. TheSYN_ACK also contains the server's ISN value. The client then confirmsthe reception of the SYN_ACK with a SYN_ACK_ACK packet containing theacknowledged ISN of the server from the SYN_ACK packet that alsoincludes the token T again. The confirmations of ISNs and token T serveas a measure to ensure both parties are participating in thecommunication and seeing all of the previous packets. The token (in theHTTP response, SYN and SYN_ACK_ACK packets) also facilitates reuse ofthe connection for other HTTP requests between this particular clientand the server. Once the handshake is completed in this manner, theUDP-based connection is considered established; the server then startssending data packets with (possibly FEC-encoded) content. The content isthe same as would have been in the original HTTP response body if UDP isnot used.

The foregoing has outlined some of the more pertinent features of thesubject matter. These features should be construed to be merelyillustrative. Many other beneficial results can be attained by applyingthe disclosed subject matter in a different manner or by modifying thesubject matter as will be described.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the subject matter and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating a known distributed computersystem configured as a content delivery network (CDN);

FIG. 2 is a representative CDN edge machine configuration;

FIG. 3 is a diagram illustrating the basic hybrid HTTP/UDP data deliveryapproach of this disclosure; and

FIG. 4 is a diagram illustrating the operating scenario for the hybridHTTP/UDP delivery technique of this disclosure in additional detail.

DETAILED DESCRIPTION

FIG. 1 illustrates a known distributed computer system that (asdescribed below) is extended by the techniques herein.

In a known system, such as shown in FIG. 1, a distributed computersystem 100 is configured as a CDN and is assumed to have a set ofmachines 102 a-n distributed around the Internet. Typically, most of themachines are servers located near the edge of the Internet, i.e., at oradjacent end user access networks. A network operations command center(NOCC) 104 manages operations of the various machines in the system.Third party sites, such as web site 106, offload delivery of content(e.g., HTML, embedded page objects, streaming media, software downloads,and the like) to the distributed computer system 100 and, in particular,to “edge” servers. Typically, content providers offload their contentdelivery by aliasing (e.g., by a DNS CNAME) given content providerdomains or sub-domains to domains that are managed by the serviceprovider's authoritative domain name service. End users that desire thecontent are directed to the distributed computer system to obtain thatcontent more reliably and efficiently. Although not shown in detail, thedistributed computer system may also include other infrastructure, suchas a distributed data collection system 108 that collects usage andother data from the edge servers, aggregates that data across a regionor set of regions, and passes that data to other back-end systems 110,112, 114 and 116 to facilitate monitoring, logging, alerts, billing,management and other operational and administrative functions.Distributed network agents 118 monitor the network as well as the serverloads and provide network, traffic and load data to a DNS query handlingmechanism 115, which is authoritative for content domains being managedby the CDN. A distributed data transport mechanism 120 may be used todistribute control information (e.g., metadata to manage content, tofacilitate load balancing, and the like) to the edge servers.

As illustrated in FIG. 2, a given machine 200 comprises commodityhardware (e.g., an Intel Pentium processor) 202 running an operatingsystem kernel (such as Linux or variant) 204 that supports one or moreapplications 206 a-n. To facilitate content delivery services, forexample, given machines typically run a set of applications, such as anHTTP (web) proxy 207, a name server 208, a local monitoring process 210,and distributed data collection process 212. More generally, the proxy207 is software, comprising program instructions, which instructions areheld in memory and executed by a processor (or multiple processors) asneeded.

For streaming media, the CDN machine typically includes one or moremedia servers, such as a Windows Media Server (WMS) or Flash server, asrequired by the supported media formats. An alternative to usingdedicated media servers uses HTTP as the transport protocol.Architectures for HTTP-based live stream and VOD-based delivery aredescribed in U.S. Publication No. 20110173345. That approach isimplemented within a CDN and includes the high level functions ofrecording the content stream to be delivered using a recording tier, andplaying the stream using a player tier. The step of recording the streamincludes a set of sub-steps that begins when the stream is received at aCDN entry point in a source format. The stream is then converted into anintermediate format (IF), which is an internal format for delivering thestream within the CDN and comprises a stream manifest, a set of one ormore fragment indexes (FI), and a set of IF fragments. The playerprocess begins when a requesting client is associated with a CDN HTTPproxy, such as the ghost-based proxy. In response to receipt at the HTTPproxy of a request for the stream or a portion thereof, the HTTP proxyretrieves (either from the archive or the data store) the streammanifest and at least one fragment index. Using the fragment index, theIF fragments are retrieved to the HTTP proxy, converted to a targetformat, and then served in response to the client request. The sourceformat may be the same or different from the target format. Preferably,all fragments are accessed, cached and served by the HTTP proxy viaHTTP. In another embodiment, a method of delivering a stream on-demand(VOD) uses a translation tier (in lieu of the recording tier) to managethe creation and/or handling of the IF components.

A CDN edge server is configured to provide one or more extended contentdelivery features, preferably on a domain-specific, customer-specificbasis, preferably using configuration files that are distributed to theedge servers using a configuration system. A given configuration filepreferably is XML-based and includes a set of content handling rules anddirectives that facilitate one or more advanced content handlingfeatures. The configuration file may be delivered to the CDN edge servervia the data transport mechanism. U.S. Pat. No. 7,111,057 illustrates auseful infrastructure for delivering and managing edge server contentcontrol information, and this and other edge server control informationcan be provisioned by the CDN service provider itself, or (via anextranet or the like) the content provider customer who operates theorigin server.

Because the CDN infrastructure is shared by multiple third parties, itis sometimes referred to herein as a multi-tenant shared infrastructure.The CDN processes may be located at nodes that are publicly-routable onthe Internet, within or adjacent nodes that are located in mobilenetworks, in or adjacent enterprise-based private networks, or in anycombination thereof.

An overlay network web proxy (such as proxy 207 in FIG. 2) that ismetadata-configurable is sometimes referred to herein as a global hostor GHost process.

The CDN may include a storage subsystem, such as described in U.S. Pat.No. 7,472,178, the disclosure of which is incorporated herein byreference.

The CDN may operate a server cache hierarchy to provide intermediatecaching of customer content; one such cache hierarchy subsystem isdescribed in U.S. Pat. No. 7,376,716, the disclosure of which isincorporated herein by reference.

The CDN may provide secure content delivery among a client browser, edgeserver and customer origin server in the manner described in U.S.Publication No. 20040093419. Secure content delivery as describedtherein enforces SSL-based links between the client and the edge serverprocess, on the one hand, and between the edge server process and anorigin server process, on the other hand. This enables an SSL-protectedweb page and/or components thereof to be delivered via the edge server.

As an overlay, the CDN resources may be used to facilitate wide areanetwork (WAN) acceleration services between enterprise data centers(which may be privately-managed) and third party software-as-a-service(SaaS) providers.

The CDN may also provide client-side software to facilitate variouscontent delivery functions, such as HD streaming. In one approach, theclient comprises two components. The first is a non-persistent,in-browser media player that may be based upon third party (e.g., AdobeFlash or Microsoft Silverlight) components to facilitate adaptive ordynamic bitrate delivery of HD content. The second is an interface inthe form of a persistent daemon that can perform “client-side” contentdelivery. Client-side delivery means downloading content both from CDNedge servers and from other end users who are running the interface andhave previously downloaded the same content.

For a typical HTTP-based streaming use case, the client resolves a name(e.g., domain name, hostname, etc.) through DNS to an IP address andmakes a TCP connection to the server associated with that address. Onceestablished, the client sends an HTTP GET request for the desiredcontent, e.g., a media fragment. The server responds with the contentdata in a standard HTTP response body. The connection is kept open forfurther requests and responses.

With the above as background, the subject matter of this disclosure isnow described.

Hybrid HTTP+UDP Approach to Streaming and Downloads

This disclosure is a new approach to delivering media and other contentover the Internet and replaces or complements standard HTTP-baseddelivery. The main difference compared to existing standard streamingand downloads approaches is the combination of HTTP-based request andUDP-based response.

FIG. 3 is a diagram illustrating the basic approach of this disclosure.As used in this drawing, server 300 refers to the server-side of thecommunication, and the client 302 refers to the client-side. The serverand client components have been configured to execute the HTTPrequest-UDP response functionality of this disclosure. In this examplescenario, the client 302 makes HTTP GET requests (one of which is shownat 304) for given byte ranges of the content of interest, e.g.,fragments of a video, software download, or other content. Each HTTPrequest includes a HTTP header (or an equivalent out-of-band datastructure) that instructs the server code that the client is enabled toreceive a UDP response. As illustrated, at step 306 the server 300responds to the HTTP GET request with an HTTP response that includesHTTP headers and a body. This response, however, differs from aconventional HTTP response in several respects. In particular, the HTTPresponse headers include an identification of an UDP IP:port where theserver listens for UDP connections, and, optionally, a data string suchas a secret token T1. The HTTP response body, however, is empty (i.e.,it is devoid of (does not include) the content requested by the HTTP GETrequest). The first two communications occur via TCP. According to thisdisclosure, the content that would have been included in the HTTP bodyis then delivered over UDP by the subsequent UDP-based connectionillustrated. In particular, at step 308 the client, having received theHTTP response, issues a SYN to the server's UDP IP:port as identified inthe HTTP response. The SYN may include the secret token T1 (for securitypurposes). At step 310 the server returns to the client a SYNACK thatincludes a secret token T2. At step 312 the client responds with an ACKthat includes the secret token T2. The tokens ensure that only theclient receiving the original HTTP response can receive the originalcontent that was requested. At step 314 the server then responds withthe requested data packets, delivered over UDP.

Preferably, the HTTP connection is kept-alive (maintained persistently)to receive one or more additional HTTP requests (typically foradditional byte ranges of the same or different content), with therequested content being delivered over UDP in this manner.

An optional improvement is to use forward error correction (FEC) orother encoding of packets (for the UDP delivery).

The following provides additional details of a preferred embodiment ofthe hybrid HTTP/UDP protocol. FIG. 4 illustrates this hybrid HTTP/UDPconnection initialization and use scenario. As noted above, the hybridapproach differs by adding a separate channel for content transmissionover a UDP protocol. In the approach herein, the client 400 resolves aname through DNS (e.g., an overlay network authoritative DNS) to an IPaddress and makes a TCP connection to an edge server 402 in thatnetwork. Once established, at step 404 the client sends an HTTP GETrequest for the desired content, e.g., a media fragment, and includes anHTTP header (or the like) indicating that it wishes to use a UDPconnection for the transfer of the body of the response. The client mayalso include any preferences about the UDP delivery (e.g., forward errorcorrection (FEC) or other parameters), but typically it is up to theserver to decide whether to follow them. The server, assuming metadataprocessing for the object enabled usage of the UDP protocol, responds atstep 406 with an empty body (Content-Length: 0) answer. The responsepreferably also includes various headers to prevent proxies on the wayfrom caching the response. In the response headers, the client findsinformation on how to connect to the same server over UDP protocol (oralso other extra alternate server references, e.g., for multi-sourcecontent acquisition). This information typically includes an IP address(because, for example, the connection so far may have been through anexplicit proxy connection and the client may not have visibility intothe server IP address it is communicating with). The returnedinformation also includes a UDP port number, optionally some otherspecial port number (to identify which flow the communication belongsto, because there may be many such connections on the same server's UDPport), version, and token T as described above.

Once this information is received from the server 402, the client 400initiates the UDP connection at step 408 by sending a SYN UDP packet tothe server based on the information provided in the HTTP responseheaders (IP address, UDP port number, other port number, etc.). Thisrequest preferably also includes a token T from the HTTP response. Usingthe token, the server thus can verify that the client sending the UDPpacket saw the previous HTTP response and that such SYN packet should beprocessed (all unexpected packets received by the server are otherwisedropped silently without any response to the sender). As illustrated,preferably the SYN also contains the client's Initial Sequence Number(ISN1). At step 410, the server 402 responds with a SYN_ACK packet,which includes the acknowledged ISN1 value from the client's SYN packet.The SYN_ACK also contains the server's ISN value, ISN2. At step 412, theclient then confirms the reception of the SYN_ACK with a SYN_ACK_ACKpacket containing the acknowledged ISN of the server (i.e., ISN2) fromthe SYN_ACK packet that also includes the token T again. Theconfirmations of ISNs and token T serve as a measure to ensure bothparties are participating in the communication and seeing all of theprevious packets. The token (in the HTTP response, SYN and SYN_ACK_ACKpackets) facilitates reuse of the connection for other HTTP requestsbetween this particular client and the server instance. Once thehandshake is completed in this manner, the UDP-based connection isconsidered established; thus, at step 414, the server then startssending data packets with (possibly FEC-encoded) content. The content isthe same as would have been in the original HTTP response body if UDP isnot used. Preferably, the server keeps sending data packets and clientreplies with acknowledgements (ACK). ACKs may be used to facilitatecongestion control. During the handshake and data delivery, the clientmay also send (e.g., at step 416) some control messages similarly toTCP, such as RST packets. The client also sends special packets tocontrol the flow of the encoded data sent by the server, in the eventFEC is used, e.g., indicating that a current FEC block was successfullydecoded and the next block should be sent by the server.

As further indicated in FIG. 4, after the initial connectionestablishment and exchange, the client 400 may request a next piece ofcontent. To that end, at step 418 the client 400 issues a GET request(e.g., on the same domain name) for the next fragment. The request maybe sent using the existing TCP connection for the HTTP request, with theexisting UDP connection being used for the response. The response isdelivered from the server 402 at step 420. This completes thedescription of the example scenario.

Typically, the protocol connection over UDP only supports one session ata time (similarly to HTTP over TCP). While the approach does not provideexplicit multiplexing per connection, many connections may work over thesame UDP port; thus, from the operating system's perspective, themultiple-connection scenario has the appearance of multiplexing. Thus,in one embodiment, multiple UDP ports (one per stack) can be used on asingle server to better utilize available CPU processing resources.

Preferably, both the TCP connection (for HTTP) and UDP connections arekept alive after satisfying a request. Then, when another URL object hasto be downloaded (e.g., the next media fragment in a stream), both theconnections are (or can be) reused if they are still considered alive(through timeouts). Preferably, the TCP and UDP connections are notcoupled or bound in any way other than for each single HTTP GET request,each can be closed without affecting the other one, and each can bereused independently for further requests. A new HTTP GET request issent over the TCP connection, indicating via HTTP headers that anexisting UDP connection shall be reused, and the server immediatelyresponds by both sending the HTTP (empty) response over TCP and bysending the body of the requested object over the existing UDPconnection.

Before processing the new UDP packets for the new fragment, the clienttypically must get the complete HTTP response first, as (whenFEC-encoded data is used) it contains all the details necessary todecode the FEC-encoded data coming over the protocol. It is alsopossible to send a new HTTP GET request (and request reuse of theexisting connection for the data transfer) before the current datatransfer completes. This leads to ceasing existing transfer, and the newrequest is satisfied right away. There is no extra confirmation fromeach side needed for this because every data packet conforming to thisprotocol contains an ID of the HTTP response which it “belongs to”;thus, the data packets belonging to the new HTTP request are easilyidentified by the client.

Preferably, protocol-specific information is exchanged between serverand client through transport HTTP header values that conform to therequirements of HTTP/1.1. A special user-agent (UA) string may also beincluded in a client HTTP request (from a client that is enabled toimplement the protocol). As noted above, if the protocol is to be used,the HTTP response from the server contains an empty body and has theadditional transport header with details on how to make (or reuse) theconnection. More specifically, the server parses the transport headerand responds to the request by sending one of the following: an HTTPresponse with an HTTP status code 202 in case the hybrid protocol is tobe used (and the header contains the connection endpoint and tokendetails), an HTTP response with a status code 202 in case the protocolis to be used (and the header above does not contain the endpoint andtoken keys meaning that an existing protocol connection is to be reused,as requested by the client in the request header), or an HTTP responsewith any status code that it would use if the protocol is not to beused, i.e., the server will behave according to the HTTP specificationand the response contains the resource data in the body as if notransport header identifying the protocol is present. In both the 202response code cases, the response is uncacheable by proxy servers, whilethe third one is cacheable (as controlled by the usual server metadatasettings). Anti-caching headers that are imposed by the server (in theHTTP 202 response) include, for example, Pragma: no-cache,Cache-Control: private, must-revalidate, Date, Expires, Cache-Control:no-store, no-cache, max-age=0, and Vary: “*”.

The token is optional. It may be a 128-bit unsigned integer representedas 32 upper-case hexadecimal digits. The endpoint is a key whosepresence indicates to the client that it should establish a newconnection (via the protocol) to receive the content. The format for theendpoint may be:[IP_address]-[UDP_port_number]-[destination_port_number].

In a variant embodiment, the UDP delivery may be multicast UDP. In thisvariant, the client makes an HTTP request to the server, and the HTTPserver responds with a response containing an empty body, as previouslydescribed. The HTTP response also contains the transport header for thehybrid protocol, and in this case the header includes a multicast groupaddress. The client then joins the multicast group and receives therequested content. Joining the multicast group and receiving themulticast UDP packets can be facilitated (as an option) by AutomaticMulticast Tunneling (AMT) for clients in networks where native IPmulticast is not supported. Once the content is received, the client canleave the multicast group. Any number of requests for the same file(which could be a segment of a stream) by many clients can be satisfiedvia the same multicast group. To make this possible, the server sendsthe data of the file to the multicast group. Preferably, the data issent FEC-encoded. Thus for example, if the file is 1 MB in size, theserver keeps sending FEC packets containing information about the file'sdata (useful for decoding of the file by clients) for some preconfiguredamount of time at a preconfigured rate. For example, the server mightsend the 1 MB file at 8 Mbps (i.e. 1 MB per second) for some definedtime period. Within that time period all clients who request the samefile all use the same multicast group. This approach saves resources andcan support delivery of popular content (live streams or softwareupdates) to many receivers while sending a single copy of it overmulticast. The duration for which the file is being sent to themulticast group can be dynamically extended based on demand for the fileby clients; i.e., as long as there are requests for the file, the serverkeeps sending it to the multicast channel. Clients can join themulticast group at any time and, due to the FEC encoding, they only needto join for as long as necessary to receive sufficient data to decodethe original file. As an optimization, servers may send the same file'scontent on multiple multicast groups in parallel and clients may joinmultiple of them (all would be returned in the original HTTP response'sheader) to better utilize their download capacity and to receive thefiles faster (as FEC allows combining data from multiple sourceseasily).

As can be seen, the approach works by a client (configured to use thehybrid HTTP-UDP protocol) making an HTTP request to a server (configuredto use the protocol), which then returns over UDP the media or filecontent as a response to the HTTP request. This approach enables aprovider to use or leverage standard CDN infrastructure available forHTTP including access control, authentication, QoS reporting,GEO-location and other features built into an HTTP-based webserver,while providing complete control of the media or file data transmissionon top of UDP. The latter includes, without limitation, customcongestion control approaches and/or encoding of the packets such asforward error correction (as described above) to provide betterexperience and control even when networking characteristics (such aspacket loss and latency) are not optimal. In such case, the approachthus combines the benefits of using existing features of HTTP fromtraditional downloads and streaming solutions with the benefits ofUDP-based transmission of data. The hybrid approach is very advantageousbecause it uses existing HTTP-based solutions for controlling what isgoing to be transmitted but at the same time benefits from usingUDP-based data transmission for the media or other content. In caseswhere UDP communication is not possible, standard HTTP is used for bothrequest and response to ensure every client can receive the data. Thesupport for UDP is communicated via HTTP.

An application and server can use the approach directly or a proxy couldbe used to add support for this functionality. For example, a streamingapplication or a browser (for downloads) can make a request to orthrough a proxy over HTTP only and the proxy then uses this approachwhen communicating with a server. Preferably, client-to-proxycommunication is HTTP-only while the proxy communicates with the serverover hybrid HTTP and UDP-based protocol. Similarly, a web server can usea proxy (either transparently or explicitly) to add support for thisapproach.

For streaming, there is an added benefit in that the approach works wellwith any existing HTTP-based streaming technologies.

More generally, the approach described herein may be implementedwhenever HTTP streaming from server to client is required. A clientrefers to the side receiving the video stream and playing it back to ascreen. Clients may be running on a desktop computer, mobile device(smart phones, tablets, etc.), set-top boxes, smart televisions, or anyother connected devices. The server is the side sending out the streamdata to the client. As referenced herein, HTTP-based streaming refers toany video streaming which uses only HTTP protocol to request anddownload objects to construct the stream for playback. Non-limitingexamples include Apple HLS, Adobe HDS, Microsoft SmoothStreaming, andMPEG-DASH. The approach herein may be used to support live and on-demandstreaming, e.g., an MPEG-2 transport stream containing a program streammade up of an H.264 video elementary stream, and an AAC audio elementarystream. This example is not intended to be limiting of course.

The HTTP/UDP hybrid delivery approach described herein improves videostreaming quality. The approach can fit transparently into existingHTTP-based streaming solutions. The hybrid approach significantlyreduces the negative impact of packet loss and latency for media datatransfer, thereby significantly improving streaming quality.

The client part of the technology described herein may be implemented asa software development kit (SDK) for one or more platforms (e.g., iOS,Android, etc.), and the SDK may be used in applications on the platformsto enable compatible mobile apps to download/stream content using thehybrid HTTP/UDP approach. Thus, for example, an application either canuse the SDK to download URL objects via its API or it may use a localHTTP server created by the SDK to stream/download from.

The hybrid protocol provides significant advantages, especially whencoupled with other technologies such as FEC-based UDP delivery andadvanced congestion control techniques. FEC is a well-known techniquethat overcomes packet loss by utilizing redundancy. In FEC, instead ofsending an original flow of packets (where each packet represents apiece of the data), the sender encodes the data into a new set ofpackets (where each packet holds information about various parts oforiginal data with additional parity data). The new set is generallylarger in size compared to the original data. The receiver of theencoded data performs decoding and, once it receives enough codedpackets, reconstructs the original data. To reconstruct the data usuallyonly the same amount of encoded data as the original data size needs tobe received, or slightly more. This allows for the transfer to be lossywhile still being able to decode the original data without any need forretransmission of the data. There are various FEC codes that aresuitable for different loss rates and applications. IETF ReliableMulticast Working Group has specified three FEC schemes to be used fordelivering content reliably, namely, RaptorQ (as described in RFC 6330),Low-density parity-check (LDPC) (as described in RFC 5170), andReed-Solomon (as described in RFC 5510). Other known FEC techniquesinclude Raptor R10 (as described in RFC 5053), which is an older versionof RaptorQ. Congestion control schemes, such as Akamai® FastTCP™, may beadapted for use to facilitate custom congestion control for the datapackets that are sent from the server to the client. Generally, thisknown scheme uses queuing delay to measure network congestion. At thesender, congestion window (CWND) is used to control the number of bytesto send. Initially, and according to the scheme, CWND is increasedquickly to match up the transmission delay, then adjusted lessaggressively to match up the variation of queuing delay (as packets arequeued along the path). The sender adjusts its CWND to adapt to networkchanges.

The technique described herein provides for increase in total throughputunder random environmental (non-congestive) loss and high latency,compared to standard TCP or multiple parallel TCP connections. Thetechnique also provides for increase in stability of throughput via moreaccurate bandwidth estimation and advanced congestion detection.Further, the technique provides for reduction in start-up times,reduction in rebuffering events of quantity and duration, and reductionin frequency of bitrate switching. All of the above lead to longerviewing times and better user experience. The described technique alsoprovides for smooth fail-back mode using standard HTTP approach forstreaming.

As a variant, the UDP-based portion of the delivery may be carried outvia multipath. In general, multipath refers to downloading from multiplesources (e.g., multiple edge servers, end user peers, or the like) inparallel. Multipath also refers to the scenario wherein the same server(or peer) is used for downloading multiple times in parallel overdifferent paths in the network. Such schemes may be useful when FECcoding is implemented, as FEC allows for combining data from manysources without any reconciliation. In certain circumstances,downloading from multiple sources in parallel (and even dynamicallychanging those sources depending on their performance) may beimplemented during with respect to the UDP phase.

In a representative implementation, the subject functionality isimplemented in software, as computer program instructions executed by aprocessor.

More generally, the techniques described herein are provided using a setof one or more computing-related entities (systems, machines, processes,programs, libraries, functions, or the like) that together facilitate orprovide the functionality described above. In a typical implementation,a representative machine on which the software executes comprisescommodity hardware, an operating system, an application runtimeenvironment, and a set of applications or processes and associated data,that provide the functionality of a given system or subsystem. Asdescribed, the functionality may be implemented in a standalone machine,or across a distributed set of machines. The functionality may beprovided as a service, e.g., as a SaaS solution.

While the above describes a particular order of operations performed bycertain embodiments of the invention, it should be understood that suchorder is exemplary, as alternative embodiments may perform theoperations in a different order, combine certain operations, overlapcertain operations, or the like. References in the specification to agiven embodiment indicate that the embodiment described may include aparticular feature, structure, or characteristic, but every embodimentmay not necessarily include the particular feature, structure, orcharacteristic.

While the disclosed subject matter has been described in the context ofa method or process, the subject disclosure also relates to apparatusfor performing the operations herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer. Such a computer program may bestored in a computer readable storage medium, such as, but is notlimited to, any type of disk including an optical disk, a CD-ROM, and amagnetic-optical disk, a read-only memory (ROM), a random access memory(RAM), a magnetic or optical card, or any type of media suitable forstoring electronic instructions, and each coupled to a computer systembus.

While given components of the system have been described separately, oneof ordinary skill will appreciate that some of the functions may becombined or shared in given instructions, program sequences, codeportions, and the like.

Preferably, the functionality is implemented in an application layersolution, although this is not a limitation, as portions of theidentified functions may be built into an operating system or the like.

The functionality may be implemented with other application layerprotocols besides HTTP, such as HTTPS, or any other protocol havingsimilar operating characteristics.

Generalizing, there is no limitation on the type of computing entitythat may implement the client-side or server-side of the connection. Anycomputing entity (system, machine, device, program, process, utility, orthe like) may act as the client or the server.

What is claimed is as follows:
 1. A method of data delivery, comprising:at a server executing on a hardware platform: receiving, from a clientand over an HTTP over TCP connection, an HTTP request that identifies acontent object; returning to the client over the HTTP over TCPconnection a response to the HTTP request that includes at least aheader and a response body, the header including an IP address and UDPport number associated with the server, the response body being devoidof the content object requested; receiving from the client over a UDPconnection a UDP request directed to the IP address and UDP port number;and upon a successful verification of the UDP request, returning aresponse to the UDP request over the UDP connection and that includesthe content object that was identified in the HTTP request but notreturned in the response body, wherein: the HTTP over TCP connection iskept-alive after satisfying the HTTP request; the UDP connection iskept-alive after satisfying the UDP request; keeping alive the HTTP overTCP connection and the UDP connection enables connection re-use for oneor more additional HTTP requests for additional byte ranges of contentthat includes the content object, with the additional byte ranges beingdelivered over the UDP connection; and the delivery over the UDPconnection being in accordance with a UDP delivery preference.
 2. Themethod as described in claim 1 wherein the HTTP request comprises anHTTP request header that includes an indication that the client desiresto use a UDP connection for transfer of a body of the response to theHTTP request, together with the UDP delivery preference.
 3. The methodas described in claim 1 wherein the HTTP request header also includesinformation to prevent one or more proxies between the server and theclient from caching the response to the HTTP request.
 4. The method asdescribed in claim 1 wherein the content object is returned to theclient according to a forward error correction (FEC) coding scheme. 5.The method as described in claim 1 wherein the content object and theadditional byte ranges of content comprise one of: fragments of a video,a software download, and other media content.
 6. The method as describedin claim 1 wherein the server is further configured to drop anyunexpected packet received from the client without providing anyresponse.
 7. The method as described in claim 1 wherein the response tothe UDP request includes a multicast group address at which the clientcan obtain the content object.
 8. The method as described in claim 1:wherein the header also includes a security token, and whereinsuccessful verification of the UDP request also verifies that the UDPrequest includes the security token; and wherein the UDP requestincludes a sequence number, and wherein the response to the UDP requestincludes the sequence number.
 9. An apparatus, comprising: a hardwareprocessor; and computer memory holding computer program code executingin the hardware processor, the computer program code configured to:receiving, from a client and over an HTTP over TCP connection, an HTTPrequest that identifies a content object; returning to the client overthe HTTP over TCP connection a response to the HTTP request thatincludes a header and a response body, the header including at least anIP address and UDP port number associated with the server, the responsebody being devoid of the content object requested; receiving from theclient over a UDP connection a UDP request directed to the IP addressand UDP port number; and upon a successful verification of the UDPrequest, returning a response to the UDP request over the UDP connectionand that includes the content object that was identified in the HTTPrequest but not returned in the response body, wherein: the HTTP overTCP connection is kept-alive after satisfying the HTTP request; the UDPconnection is kept-alive after satisfying the UDP request; keeping alivethe HTTP over TCP connection and the UDP connection enables connectionre-use for one or more additional HTTP requests for additional byteranges of content that includes the content object, with the additionalbyte ranges being delivered over the UDP connection; and the deliveryover the UDP connection being in accordance with a UDP deliverypreference.
 10. The apparatus as described in claim 9 wherein the HTTPrequest comprises an HTTP request header that includes an indicationthat the client desires to use a UDP connection for transfer of a bodyof the response to the HTTP request, together with the UDP deliverypreference.
 11. The apparatus as described in claim 9 wherein the HTTPrequest header also includes information to prevent one or moredownstream proxies from caching the response to the HTTP request. 12.The apparatus as described in claim 9 wherein the content object isreturned to the client according to a forward error correction (FEC)coding scheme.
 13. The apparatus as described in claim 9 wherein thecontent object and the additional byte ranges of content comprise oneof: fragments of a video, a software download, and other media content.14. The apparatus as described in claim 9 wherein computer program codeis further configured to drop any unexpected packet received from theclient without providing any response.
 15. The apparatus as described inclaim 9 wherein the response to the UDP request includes a multicastgroup address at which the client can obtain the content object.
 16. Theapparatus as described in claim 9: wherein the header also includes asecurity token, and wherein successful verification of the UDP requestalso verifies that the UDP request includes the security token; andwherein the UDP request includes a sequence number, and wherein theresponse to the UDP request includes the sequence number.